1. Use the DISCUSSION tab above to find your discussion questions and/or tasks.
2. POST YOUR GROUP CONSENSUS HERE (The group leader for these 4 Binary Choice activities will be the member whose first name comes earliest in alphabetical order.):
Binary Choice Item # 1 - flaw(s)? The use of the word "they" is unclear. The reader is unsure if "they" refers to the teachers or district administrators. Using the pronoun “they” in the statement is not specific enough. Either “district administrator,” “teacher” or “both district administrators and teachers” should replace the word “they.” Evidence: Commandment 2: This is definitely an example of commandment 2. The use of the word "they" is ambiguous and can refer to either district administrators or teachers. Commandment 2 says Thou shall not employ ambiguous statements in your assessment items. Just because you know what you mean does not necessarily mean that your students will have the same understanding. It is very important to realize that sentences where pronouns are used can be confusing as to whom the unclear as to whom the pronoun is referring to. Binary Choice Item # 2 - flaw(s)? A majority of these words in this statement/sentence would be considered advanced for the reader. A large amount of these words are unfamiliar and it would be a very big distraction and may result with incorrect answers. Words such as “predilections” and “idiosyncratic proclivities” are words that are not suitable for a typical student. Evidence:
In commandment # 5 it states, "Thou shall not use vocabulary that is more advanced than your audience." It is important to use vocabulary words that the students are familiar with. It is important to use vocabulary that is suitable for the students that will be taking your test.
Binary Choice Item # 3 - flaw(s)?: This is very tricky. More than one concept seems to be the most logical answer. It almost seems this could be split into two questions at the comma. Also just because both statements refer to student attitudes does not mean they are the same concept. One is discussing attitudinal assessments as becoming commonplace and the other is describing them as being ineffective. Evidence: This is under the Item Writing Guidelines, "Avoid items that include more than one concept. When read it gives the impression that the assessment of students' attitudes is a different concept from individual student's attitudinal status. Maybe this statement falls under the “Item Writing Guidelines for Binary-Choice Assessments” section of the PowerPoint, which discusses the use of negatives. The phrase “almost impossible” may be the use of a negative. This may fall into the category of commandment 4, Thou shall not employ complex syntax in assessment items, because it also discusses on this slide about avoiding complicated sentence construction and to use very simple sentences.
Binary Choice Item # 4 - flaw(s)? It could be use of a double negative with the use of the word "fail". It is avoiding failing to discriminate by not creating questions that fail to elicit incorrect answers. This sentence is so difficult to understand is because it uses several negatives. If written in your own words it may sound like this... To avoid making tests that do not recognize the differences among test takers, test questions should never be made that do not have a reasonable amount of incorrect responses from students. The fact that the sentence had to be recreated means that it was too complex.
Evidence: It goes under one of the 5 Commandments. "Thou shall not employ complex syntax in assessment items." Since it is a complicated sentence, it is hard to understand what the sentence actually means. Also, the word "NOT" is used twice, which provides evidence of the use of negatives in this sentence. 3. POST YOUR GROUP PRODUCT HERE - Multiple Binary-Choice assessment item (the group member whose birthday is closest to Christmas will be the leader for this Multiple Binary-Choice activity :
4. POST YOUR GROUP CONSENSUS HERE (The group leader for these 4 Multiple Choice activities will be the member whose last name comes earliest in alphabetical order.):
Multiple-Choice Item # 1 - flaw(s)?The use of the word not is not necessary and is misused in the question. Using a negative such as not can make a question confusing and not provide a fair chance to answer the question. If a negative is to be used it should be highlighted in some way. Also, this question does not a self contained question or problem; the information that the question is asking for is vague and the stem of this problem is too short in comparison with the alternative.
Evidence:
Under Item Writing Guidelines for Multiple Choice Assessments it is clearly stated that as a rule, negatively stated stems should be avoided. "If there is a compelling reason for using a negative in the stem be sure to highlight with italics, boldface type or underlining". Also the guidelines state that a stem should consist of a self contained question or problem in which a lengthy stem and short alternatives are preferred over short stems and long alternatives.
Multiple-Choice Item # 2 - flaw(s)?The use of the word "an" at the end of the question gives a clear choice in the correct answer which, grammatically speaking, could only be "A".
Evidence:Commandment number three states: "thou shall not provide students with unintended clues regarding appropriate responses". As shown in the example the use of the word "an" grammatically makes only one answer correct.
Multiple-Choice Item # 3 - flaw(s)?
The evident flaw is using "all-of-the-above". When this is used a student who knows only two of the alternatives are correct can guess that the other two are correct as well.It should only be used when you make a test based inference about student knowledge.
Evidence:The writing guidelines for multiple-choice assessments state that teachers should carefully consider using alternative "all of the above" because if a student knows only two of the alternatives are correct then they can guess that the other responses are correct as well. Therefore, the student does not need to know ALL of the information about this question to answer it, they only need to know SOME of it.
Multiple-Choice Item # 4 - flaw(s)? The length of item "D" in this question is considerably longer than the other three items giving an unintended clue as to what the correct answer is. More effort was provided in writing item "D" suggesting that there is something special about it.
Evidence:
The writing guidelines for multiple-choice assessments state that "item writers frequently fail to realize that the length of a multiple-choice item can give an unintended clue to the correct answer." Furthermore, "students can figure that if the teacher has given so much attention to constructing an answer, there must be something special about it. Looking at the four alternatives above, it is obvious that "D" is significantly longer than the other alternatives. Special attention was given to this alternative.
5. POST YOUR GROUP CONSENSUS HERE (The group leader for this Matching Item activity will be the member whose birthday is closest to Valentine's Day.):
“Find the flaws . . . use evidence from the ‘5 Commandments and/or the Item Writing Guidelines to support your answers.”
On the line to the left of each measurement concept listed in Column A, please write the letter of the most relevant measurement procedure in Column B.
1. Use the DISCUSSION tab above to find your discussion questions and/or tasks.
2. POST YOUR GROUP CONSENSUS HERE (The group leader for these 4 Binary Choice activities will be the member whose first name comes earliest in alphabetical order.):
Binary Choice Item # 1 - flaw(s)?
The use of the word "they" is unclear. The reader is unsure if "they" refers to the teachers or district administrators. Using the pronoun “they” in the statement is not specific enough. Either “district administrator,” “teacher” or “both district administrators and teachers” should replace the word “they.”
Evidence: Commandment 2:
This is definitely an example of commandment 2. The use of the word "they" is ambiguous and can refer to either district administrators or teachers. Commandment 2 says Thou shall not employ ambiguous statements in your assessment items. Just because you know what you mean does not necessarily mean that your students will have the same understanding. It is very important to realize that sentences where pronouns are used can be confusing as to whom the unclear as to whom the pronoun is referring to.
Binary Choice Item # 2 - flaw(s)?
A majority of these words in this statement/sentence would be considered advanced for the reader. A large amount of these words are unfamiliar and it would be a very big distraction and may result with incorrect answers. Words such as “predilections” and “idiosyncratic proclivities” are words that are not suitable for a typical student.
Evidence:
In commandment # 5 it states, "Thou shall not use vocabulary that is more advanced than your audience." It is important to use vocabulary words that the students are familiar with. It is important to use vocabulary that is suitable for the students that will be taking your test.
Binary Choice Item # 3 - flaw(s)?:
This is very tricky. More than one concept seems to be the most logical answer. It almost seems this could be split into two questions at the comma. Also just because both statements refer to student attitudes does not mean they are the same concept. One is discussing attitudinal assessments as becoming commonplace and the other is describing them as being ineffective.
Evidence:
This is under the Item Writing Guidelines, "Avoid items that include more than one concept. When read it gives the impression that the assessment of students' attitudes is a different concept from individual student's attitudinal status. Maybe this statement falls under the “Item Writing Guidelines for Binary-Choice Assessments” section of the PowerPoint, which discusses the use of negatives. The phrase “almost impossible” may be the use of a negative. This may fall into the category of commandment 4, Thou shall not employ complex syntax in assessment items, because it also discusses on this slide about avoiding complicated sentence construction and to use very simple sentences.
Binary Choice Item # 4 - flaw(s)?
It could be use of a double negative with the use of the word "fail". It is avoiding failing to discriminate by not creating questions that fail to elicit incorrect answers. This sentence is so difficult to understand is because it uses several negatives. If written in your own words it may sound like this... To avoid making tests that do not recognize the differences among test takers, test questions should never be made that do not have a reasonable amount of incorrect responses from students. The fact that the sentence had to be recreated means that it was too complex.
Evidence:
It goes under one of the 5 Commandments. "Thou shall not employ complex syntax in assessment items." Since it is a complicated sentence, it is hard to understand what the sentence actually means. Also, the word "NOT" is used twice, which provides evidence of the use of negatives in this sentence. 3. POST YOUR GROUP PRODUCT HERE - Multiple Binary-Choice assessment item (the group member whose birthday is closest to Christmas will be the leader for this Multiple Binary-Choice activity :
*Solve for both variables.
2x + x – 5 = 34
7 + 7y – 3y = 67
Answer True or False.
A. y > x
B. x = y
C. y ≤ 15
D. 2 x = y
(Answers: <x = 13>, <y = 15>, <A = True>, <B = False>, <C = True>, <D = False>
4. POST YOUR GROUP CONSENSUS HERE (The group leader for these 4 Multiple Choice activities will be the member whose last name comes earliest in alphabetical order.):
Multiple-Choice Item # 1 - flaw(s)?The use of the word not is not necessary and is misused in the question. Using a negative such as not can make a question confusing and not provide a fair chance to answer the question. If a negative is to be used it should be highlighted in some way. Also, this question does not a self contained question or problem; the information that the question is asking for is vague and the stem of this problem is too short in comparison with the alternative .
Evidence:
Under Item Writing Guidelines for Multiple Choice Assessments it is clearly stated that as a rule, negatively stated stems should be avoided. "If there is a compelling reason for using a negative in the stem be sure to highlight with italics, boldface type or underlining". Also the guidelines state that a stem should consist of a self contained question or problem in which a lengthy stem and short alternatives are preferred over short stems and long alternatives.
Multiple-Choice Item # 2 - flaw(s)?The use of the word "an" at the end of the question gives a clear choice in the correct answer which, grammatically speaking, could only be "A".
Evidence:Commandment number three states: "thou shall not provide students with unintended clues regarding appropriate responses". As shown in the example the use of the word "an" grammatically makes only one answer correct.
Multiple-Choice Item # 3 - flaw(s)?
The evident flaw is using "all-of-the-above". When this is used a student who knows only two of the alternatives are correct can guess that the other two are correct as well.It should only be used when you make a test based inference about student knowledge.
Evidence:The writing guidelines for multiple-choice assessments state that teachers should carefully consider using alternative "all of the above" because if a student knows only two of the alternatives are correct then they can guess that the other responses are correct as well. Therefore, the student does not need to know ALL of the information about this question to answer it, they only need to know SOME of it.
Multiple-Choice Item # 4 - flaw(s)? The length of item "D" in this question is considerably longer than the other three items giving an unintended clue as to what the correct answer is. More effort was provided in writing item "D" suggesting that there is something special about it.
Evidence:
The writing guidelines for multiple-choice assessments state that "item writers frequently fail to realize that the length of a multiple-choice item can give an unintended clue to the correct answer." Furthermore, "students can figure that if the teacher has given so much attention to constructing an answer, there must be something special about it. Looking at the four alternatives above, it is obvious that "D" is significantly longer than the other alternatives. Special attention was given to this alternative.
5. POST YOUR GROUP CONSENSUS HERE (The group leader for this Matching Item activity will be the member whose birthday is closest to Valentine's Day.):
“Find the flaws . . . use evidence from the ‘5 Commandments and/or the Item Writing Guidelines to support your answers.”
On the line to the left of each measurement concept listed in Column A, please write the letter of the most relevant measurement procedure in Column B.
Matching Item # 1 - flaw(s)?
Evidence:**